Lab 10 - Vanilla CNN and Fine-Tune VGG16 - for Dogs and Cats Classification¶
NAME - LAKSHAY SONI
STUDENT_ID - 8888349
Asirra: The Dogs vs. Cats Dataset
Published in 2007 by Microsoft. This dataset appears in Chollet, Deep Learing with Python 2nd edition, chapter 8 (book and associated notebook). The dataset can be obtained from:
Dogs versus Cats - The Kaggle competition The Dataset (with an Open Data license) See the competition page for a nice introduction to the competition. Here is a description of the dataset taken from there:
The Asirra data set
Web services are often protected with a challenge that's supposed to be easy for people to solve, but difficult for computers. Such a challenge is often called a CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart). CAPTCHAs are used for many purposes, such as to reduce email and blog spam and prevent brute-force attacks on web site passwords.
Asirra (Animal Species Image Recognition for Restricting Access) is a CAPTCHA that works by asking users to identify photographs of cats and dogs. This task is difficult for computers, but studies have shown that people can accomplish it quickly and accurately. Many even think it's fun! Here is an example of the Asirra interface:
Asirra is unique because of its partnership with Petfinder.com, the world's largest site devoted to finding homes for homeless pets. They've provided Microsoft Research with over three million images of cats and dogs, manually classified by people at thousands of animal shelters across the United States. Kaggle is fortunate to offer a subset of this data for fun and research.
EDA: Explore the data with relevant graphs, statistics and insights (1.5 points)
loading Necessary Modules¶
In [ ]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import plotly
import os, shutil, pathlib
import tensorflow as tf
from sklearn.metrics import confusion_matrix
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from tensorflow.python import keras
from tensorflow.python.keras.models import Sequential
from keras.layers import Dense, Conv2D, Activation, MaxPool2D, Flatten, Dropout, BatchNormalization
from keras.optimizers import RMSprop,Adam
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from keras.utils import plot_model
import math
from PIL import Image
from keras.optimizers import RMSprop
import pickle
import zipfile
Loading Dataset¶
In [ ]:
import os, shutil, pathlib
original_dir = pathlib.Path("C:/Users/HP/OneDrive/Desktop/LAKSHAY/train")
d_folder = pathlib.Path("C:/Users/HP/OneDrive/Desktop/LAKSHAY")
def make_subset(subset_name, start_index, end_index):
for category in ("cat", "dog"):
dir = d_folder / subset_name / category
os.makedirs(dir, exist_ok=True) # Use exist_ok=True to avoid FileExistsError
fnames = [f"{category}.{i}.jpg" for i in range(start_index, end_index)]
for fname in fnames:
shutil.copyfile(src=original_dir / fname,
dst=dir / fname)
make_subset("train", start_index=0, end_index=1000)
make_subset("validation", start_index=1000, end_index=1500)
make_subset("test", start_index=1500, end_index=2500)
In [ ]:
# number of images in each subset
print("Total training cat images:", len(os.listdir(d_folder / "train/cat")))
print("Total training dog images:", len(os.listdir(d_folder / "train/dog")))
print("Total validation cat images:", len(os.listdir(d_folder / "validation/cat")))
print("Total validation dog images:", len(os.listdir(d_folder / "validation/dog")))
print("Total test cat images:", len(os.listdir(d_folder / "test/cat")))
print("Total test dog images:", len(os.listdir(d_folder / "test/dog")))
Total training cat images: 1000 Total training dog images: 1000 Total validation cat images: 500 Total validation dog images: 500 Total test cat images: 1000 Total test dog images: 1000
The model isn't biased toward any one class thanks to an equal distribution of 1000 photos of dogs and cats.
Counting Cats and Dogs Images in subdataset¶
In [ ]:
from tensorflow.keras.utils import image_dataset_from_directory
train_dataset = image_dataset_from_directory(
d_folder / "train",
image_size=(180, 180),
batch_size=32)
validation_dataset = image_dataset_from_directory(
d_folder / "validation",
image_size=(180, 180),
batch_size=32)
test_dataset = image_dataset_from_directory(
d_folder / "test",
image_size=(180, 180),
batch_size=32)
Found 2000 files belonging to 2 classes. Found 1000 files belonging to 2 classes. Found 2000 files belonging to 2 classes.
Two image classes are confirmed by the data summary spanning test, validation, and perhaps training sets.
In [ ]:
# Function to visualize sample images in a grid
def visualize_samples(subset_folder, num_samples=4):
classes = os.listdir(subset_folder)
fig, axes = plt.subplots(num_samples, num_samples, figsize=(5, 5))
for i, class_name in enumerate(classes):
class_folder = os.path.join(subset_folder, class_name)
if os.path.isdir(class_folder): # Check if it's a directory
image_files = os.listdir(class_folder)[:num_samples]
for j, image_file in enumerate(image_files):
if i < num_samples and j < num_samples:
image_path = os.path.join(class_folder, image_file)
image = Image.open(image_path)
axes[i, j].imshow(image)
axes[i, j].set_title(f"Class: {class_name}")
axes[i, j].axis('off')
plt.tight_layout()
plt.show()
In [ ]:
# Visualize sample images from training set
print("Visualizing sample images from training set:")
visualize_samples(train_folder)
Visualizing sample images from training set:
In [ ]:
# Visualize sample images from validation set
print("Visualizing sample images from validation set:")
visualize_samples(val_folder)
Visualizing sample images from validation set:
In [ ]:
# Visualize sample images from test set
print("Visualizing sample images from test set:")
visualize_samples(test_folder)
Visualizing sample images from test set:
On the top we visualize some model depecting images from all of 3 subset folder Train, Test and Validation for both dogs and cats.
Defining Model¶
In [ ]:
from tensorflow.keras import layers
from tensorflow import keras
from tensorflow.keras import layers
import pathlib
from tensorflow.keras.utils import image_dataset_from_directory
import matplotlib.pyplot as plt
In [ ]:
inputs = keras.Input(shape=(180, 180, 3))
x = layers.Rescaling(1./255)(inputs)
x = layers.Conv2D(filters=32, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=64, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=128, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.Flatten()(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs=inputs, outputs=outputs)
In [ ]:
model.summary()
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 180, 180, 3)] 0
rescaling (Rescaling) (None, 180, 180, 3) 0
conv2d (Conv2D) (None, 178, 178, 32) 896
max_pooling2d (MaxPooling2D (None, 89, 89, 32) 0
)
conv2d_1 (Conv2D) (None, 87, 87, 64) 18496
max_pooling2d_1 (MaxPooling (None, 43, 43, 64) 0
2D)
conv2d_2 (Conv2D) (None, 41, 41, 128) 73856
max_pooling2d_2 (MaxPooling (None, 20, 20, 128) 0
2D)
conv2d_3 (Conv2D) (None, 18, 18, 256) 295168
max_pooling2d_3 (MaxPooling (None, 9, 9, 256) 0
2D)
conv2d_4 (Conv2D) (None, 7, 7, 256) 590080
flatten (Flatten) (None, 12544) 0
dense (Dense) (None, 1) 12545
=================================================================
Total params: 991,041
Trainable params: 991,041
Non-trainable params: 0
_________________________________________________________________
The model we explained above consists of several layers, such as pooling and convolutional layers.
It accepts a picture with dimensions of 180x180x3 as input.
A total of 991,041 trainable parameters make up the model.
The trainable parameter is used to specify whether a layer or a set of layers should be trainable during the training process. (Citing for the defination https://keras.io/guides/transfer_learning/)
Define a Neural Network of your choice (0.5 points)
In [ ]:
model.compile(loss="binary_crossentropy",
optimizer="rmsprop",
metrics=["accuracy"])
In [ ]:
callbacks = [
keras.callbacks.ModelCheckpoint(
filepath="C:/Users/HP/OneDrive/Desktop/LAKSHAY/models/t_model_1.keras",
save_best_only=True,
monitor="val_loss")
]
history = model.fit(
train_dataset,
epochs=30,
validation_data=validation_dataset,
callbacks=callbacks)
Epoch 1/30
63/63 [==============================] - 54s 847ms/step - loss: 0.7176 - accuracy: 0.5440 - val_loss: 1.0704 - val_accuracy: 0.5000 Epoch 2/30 63/63 [==============================] - 52s 828ms/step - loss: 0.6985 - accuracy: 0.5425 - val_loss: 0.6757 - val_accuracy: 0.5940 Epoch 3/30 63/63 [==============================] - 51s 810ms/step - loss: 0.6804 - accuracy: 0.5685 - val_loss: 0.6900 - val_accuracy: 0.5060 Epoch 4/30 63/63 [==============================] - 51s 813ms/step - loss: 0.6662 - accuracy: 0.6040 - val_loss: 0.6294 - val_accuracy: 0.6720 Epoch 5/30 63/63 [==============================] - 52s 820ms/step - loss: 0.6266 - accuracy: 0.6570 - val_loss: 0.5924 - val_accuracy: 0.6720 Epoch 6/30 63/63 [==============================] - 53s 842ms/step - loss: 0.5885 - accuracy: 0.6870 - val_loss: 0.5848 - val_accuracy: 0.6810 Epoch 7/30 63/63 [==============================] - 53s 843ms/step - loss: 0.5605 - accuracy: 0.7075 - val_loss: 0.5928 - val_accuracy: 0.6870 Epoch 8/30 63/63 [==============================] - 54s 848ms/step - loss: 0.5516 - accuracy: 0.7160 - val_loss: 0.5735 - val_accuracy: 0.7150 Epoch 9/30 63/63 [==============================] - 54s 851ms/step - loss: 0.5094 - accuracy: 0.7530 - val_loss: 0.5516 - val_accuracy: 0.7040 Epoch 10/30 63/63 [==============================] - 54s 857ms/step - loss: 0.4913 - accuracy: 0.7710 - val_loss: 0.5669 - val_accuracy: 0.6940 Epoch 11/30 63/63 [==============================] - 55s 867ms/step - loss: 0.4341 - accuracy: 0.7900 - val_loss: 0.5823 - val_accuracy: 0.6840 Epoch 12/30 63/63 [==============================] - 55s 874ms/step - loss: 0.3898 - accuracy: 0.8215 - val_loss: 0.5827 - val_accuracy: 0.7280 Epoch 13/30 63/63 [==============================] - 56s 883ms/step - loss: 0.3539 - accuracy: 0.8405 - val_loss: 0.5700 - val_accuracy: 0.7270 Epoch 14/30 63/63 [==============================] - 56s 888ms/step - loss: 0.2931 - accuracy: 0.8790 - val_loss: 0.7396 - val_accuracy: 0.7070 Epoch 15/30 63/63 [==============================] - 59s 931ms/step - loss: 0.2569 - accuracy: 0.8965 - val_loss: 0.7234 - val_accuracy: 0.7230 Epoch 16/30 63/63 [==============================] - 56s 880ms/step - loss: 0.2140 - accuracy: 0.9130 - val_loss: 0.7704 - val_accuracy: 0.7260 Epoch 17/30 63/63 [==============================] - 56s 896ms/step - loss: 0.1722 - accuracy: 0.9380 - val_loss: 0.7596 - val_accuracy: 0.7500 Epoch 18/30 63/63 [==============================] - 57s 895ms/step - loss: 0.1407 - accuracy: 0.9455 - val_loss: 0.8457 - val_accuracy: 0.7580 Epoch 19/30 63/63 [==============================] - 57s 910ms/step - loss: 0.0858 - accuracy: 0.9745 - val_loss: 1.3540 - val_accuracy: 0.7300 Epoch 20/30 63/63 [==============================] - 60s 954ms/step - loss: 0.1103 - accuracy: 0.9640 - val_loss: 1.0983 - val_accuracy: 0.7460 Epoch 21/30 63/63 [==============================] - 59s 938ms/step - loss: 0.0844 - accuracy: 0.9700 - val_loss: 1.2266 - val_accuracy: 0.7430 Epoch 22/30 63/63 [==============================] - 60s 956ms/step - loss: 0.0745 - accuracy: 0.9725 - val_loss: 1.4835 - val_accuracy: 0.7130 Epoch 23/30 63/63 [==============================] - 61s 970ms/step - loss: 0.0487 - accuracy: 0.9835 - val_loss: 1.3657 - val_accuracy: 0.7330 Epoch 24/30 63/63 [==============================] - 61s 974ms/step - loss: 0.0770 - accuracy: 0.9725 - val_loss: 1.3963 - val_accuracy: 0.7490 Epoch 25/30 63/63 [==============================] - 61s 966ms/step - loss: 0.0404 - accuracy: 0.9870 - val_loss: 1.4732 - val_accuracy: 0.7380 Epoch 26/30 63/63 [==============================] - 87s 1s/step - loss: 0.0377 - accuracy: 0.9880 - val_loss: 1.6684 - val_accuracy: 0.7270 Epoch 27/30 63/63 [==============================] - 86s 1s/step - loss: 0.0397 - accuracy: 0.9860 - val_loss: 1.7123 - val_accuracy: 0.7350 Epoch 28/30 63/63 [==============================] - 80s 1s/step - loss: 0.0440 - accuracy: 0.9865 - val_loss: 1.7539 - val_accuracy: 0.7520 Epoch 29/30 63/63 [==============================] - 82s 1s/step - loss: 0.0525 - accuracy: 0.9835 - val_loss: 2.2499 - val_accuracy: 0.6890 Epoch 30/30 63/63 [==============================] - 83s 1s/step - loss: 0.0309 - accuracy: 0.9905 - val_loss: 1.9716 - val_accuracy: 0.7380
In [ ]:
accuracy = history.history["accuracy"]
val_accuracy = history.history["val_accuracy"]
loss = history.history["loss"]
val_loss = history.history["val_loss"]
epochs = range(1, len(accuracy) + 1)
plt.plot(epochs, accuracy, "bo", label="Training accuracy")
plt.plot(epochs, val_accuracy, "b", label="Validation accuracy")
plt.title("Training and validation accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, loss, "bo", label="Training loss")
plt.plot(epochs, val_loss, "b", label="Validation loss")
plt.title("Training and validation loss")
plt.legend()
plt.show()
Indicating that the model is adapting to fit the training data, the training loss dramatically drops across epochs.
The model's ability to categorize photos accurately in the training set is improving, as seen by the steady improvement in training accuracy.
As for the part for further analysis we can go for data argumentation as we are taking limited number of images to perform the model so we can include techniques such as Random flips, rotations, brightness or saturation.
Referance for this: (https://research.aimultiple.com/data-augmentation-techniques/)
Fine-Tune VGG16 (pre-trained on imagenet). Make sure to use validation to test for over-fitting. Plot the appropriate graph (0.5 points)
VGG16 Architecture:
VGG16 is a convolutional neural network (CNN) architecture known for its simplicity and depth. It was developed by the Visual Geometry Group (VGG) at Oxford University and achieved state-of-the-art performance on image classification tasks when it was introduced in 2014.
Referance: https://viso.ai/deep-learning/vgg-very-deep-convolutional-networks/
In [ ]:
from tensorflow.keras.applications import VGG16
In [ ]:
conv_base = keras.applications.vgg16.VGG16(
weights="imagenet",
include_top=False)
--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In[69], line 1 ----> 1 conv_base = keras.applications.vgg16.VGG16( 2 weights="imagenet", 3 include_top=False) AttributeError: module 'tensorflow.python.keras' has no attribute 'applications'
Hence above code is running later on while making comments its mistakenly run and showing error can't take risk to run it again on the data because of it will consume more time. Printing the list of trainable weights before and after freezing
Refereance (https://github.com/CSCN8010/CSCN8010/blob/main/dl_class_notebooks/05D_fine_tuning_vgg16.ipynb)
In [ ]:
conv_base.summary()
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, None, None, 3)] 0
block1_conv1 (Conv2D) (None, None, None, 64) 1792
block1_conv2 (Conv2D) (None, None, None, 64) 36928
block1_pool (MaxPooling2D) (None, None, None, 64) 0
block2_conv1 (Conv2D) (None, None, None, 128) 73856
block2_conv2 (Conv2D) (None, None, None, 128) 147584
block2_pool (MaxPooling2D) (None, None, None, 128) 0
block3_conv1 (Conv2D) (None, None, None, 256) 295168
block3_conv2 (Conv2D) (None, None, None, 256) 590080
block3_conv3 (Conv2D) (None, None, None, 256) 590080
block3_pool (MaxPooling2D) (None, None, None, 256) 0
block4_conv1 (Conv2D) (None, None, None, 512) 1180160
block4_conv2 (Conv2D) (None, None, None, 512) 2359808
block4_conv3 (Conv2D) (None, None, None, 512) 2359808
block4_pool (MaxPooling2D) (None, None, None, 512) 0
block5_conv1 (Conv2D) (None, None, None, 512) 2359808
block5_conv2 (Conv2D) (None, None, None, 512) 2359808
block5_conv3 (Conv2D) (None, None, None, 512) 2359808
block5_pool (MaxPooling2D) (None, None, None, 512) 0
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________
In [ ]:
conv_base.trainable = False
conv_base.summary()
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, None, None, 3)] 0
block1_conv1 (Conv2D) (None, None, None, 64) 1792
block1_conv2 (Conv2D) (None, None, None, 64) 36928
block1_pool (MaxPooling2D) (None, None, None, 64) 0
block2_conv1 (Conv2D) (None, None, None, 128) 73856
block2_conv2 (Conv2D) (None, None, None, 128) 147584
block2_pool (MaxPooling2D) (None, None, None, 128) 0
block3_conv1 (Conv2D) (None, None, None, 256) 295168
block3_conv2 (Conv2D) (None, None, None, 256) 590080
block3_conv3 (Conv2D) (None, None, None, 256) 590080
block3_pool (MaxPooling2D) (None, None, None, 256) 0
block4_conv1 (Conv2D) (None, None, None, 512) 1180160
block4_conv2 (Conv2D) (None, None, None, 512) 2359808
block4_conv3 (Conv2D) (None, None, None, 512) 2359808
block4_pool (MaxPooling2D) (None, None, None, 512) 0
block5_conv1 (Conv2D) (None, None, None, 512) 2359808
block5_conv2 (Conv2D) (None, None, None, 512) 2359808
block5_conv3 (Conv2D) (None, None, None, 512) 2359808
block5_pool (MaxPooling2D) (None, None, None, 512) 0
=================================================================
Total params: 14,714,688
Trainable params: 0
Non-trainable params: 14,714,688
_________________________________________________________________
Adding an argumentation layer amd a classifier to thr convulational base.
In [ ]:
data_augmentation = keras.Sequential(
[
layers.RandomFlip("horizontal"),
layers.RandomRotation(0.1),
layers.RandomZoom(0.2),
]
)
inputs = keras.Input(shape=(180, 180, 3))
x = data_augmentation(inputs)
x = keras.applications.vgg16.preprocess_input(x)
x = conv_base(x)
x = layers.Flatten()(x)
x = layers.Dense(256)(x)
x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs, outputs)
In [ ]:
model.summary()
Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_3 (InputLayer) [(None, 180, 180, 3)] 0
sequential (Sequential) (None, 180, 180, 3) 0
tf.__operators__.getitem (S (None, 180, 180, 3) 0
licingOpLambda)
tf.nn.bias_add (TFOpLambda) (None, 180, 180, 3) 0
vgg16 (Functional) (None, None, None, 512) 14714688
flatten_1 (Flatten) (None, 12800) 0
dense_1 (Dense) (None, 256) 3277056
dropout (Dropout) (None, 256) 0
dense_2 (Dense) (None, 1) 257
=================================================================
Total params: 17,992,001
Trainable params: 3,277,313
Non-trainable params: 14,714,688
_________________________________________________________________
In [ ]:
model.compile(loss="binary_crossentropy",
optimizer="rmsprop",
metrics=["accuracy"])
callbacks = [
keras.callbacks.ModelCheckpoint(
filepath="C:/Users/HP/OneDrive/Desktop/LAKSHAY/models/VGG_model_2.keras",
save_best_only=True,
monitor="val_loss")
]
history = model.fit(
train_dataset,
epochs=50,
validation_data=validation_dataset,
callbacks=callbacks)
Epoch 1/50 63/63 [==============================] - 346s 5s/step - loss: 17.1769 - accuracy: 0.8950 - val_loss: 5.9791 - val_accuracy: 0.9600 Epoch 2/50 63/63 [==============================] - 324s 5s/step - loss: 6.8046 - accuracy: 0.9400 - val_loss: 3.2259 - val_accuracy: 0.9770 Epoch 3/50 63/63 [==============================] - 342s 5s/step - loss: 5.6331 - accuracy: 0.9540 - val_loss: 3.9578 - val_accuracy: 0.9760 Epoch 4/50 63/63 [==============================] - 346s 6s/step - loss: 5.2653 - accuracy: 0.9595 - val_loss: 4.3737 - val_accuracy: 0.9730 Epoch 5/50 63/63 [==============================] - 336s 5s/step - loss: 3.7838 - accuracy: 0.9640 - val_loss: 3.0838 - val_accuracy: 0.9740 Epoch 6/50 63/63 [==============================] - 327s 5s/step - loss: 2.5937 - accuracy: 0.9740 - val_loss: 7.1747 - val_accuracy: 0.9650 Epoch 7/50 63/63 [==============================] - 323s 5s/step - loss: 2.8004 - accuracy: 0.9695 - val_loss: 3.2310 - val_accuracy: 0.9800 Epoch 8/50 63/63 [==============================] - 334s 5s/step - loss: 2.8111 - accuracy: 0.9770 - val_loss: 2.9430 - val_accuracy: 0.9840 Epoch 9/50 63/63 [==============================] - 329s 5s/step - loss: 3.0080 - accuracy: 0.9735 - val_loss: 3.6045 - val_accuracy: 0.9780 Epoch 10/50 63/63 [==============================] - 325s 5s/step - loss: 1.8158 - accuracy: 0.9790 - val_loss: 4.5785 - val_accuracy: 0.9730 Epoch 11/50 63/63 [==============================] - 316s 5s/step - loss: 2.0696 - accuracy: 0.9780 - val_loss: 2.8862 - val_accuracy: 0.9810 Epoch 12/50 63/63 [==============================] - 305s 5s/step - loss: 2.5107 - accuracy: 0.9750 - val_loss: 4.4681 - val_accuracy: 0.9730 Epoch 13/50 63/63 [==============================] - 300s 5s/step - loss: 1.8243 - accuracy: 0.9775 - val_loss: 4.3710 - val_accuracy: 0.9720 Epoch 14/50 63/63 [==============================] - 300s 5s/step - loss: 1.8922 - accuracy: 0.9800 - val_loss: 5.0211 - val_accuracy: 0.9740 Epoch 15/50 63/63 [==============================] - 300s 5s/step - loss: 2.1169 - accuracy: 0.9775 - val_loss: 3.6342 - val_accuracy: 0.9770 Epoch 16/50 63/63 [==============================] - 301s 5s/step - loss: 1.3487 - accuracy: 0.9810 - val_loss: 4.0244 - val_accuracy: 0.9790 Epoch 17/50 63/63 [==============================] - 303s 5s/step - loss: 1.2206 - accuracy: 0.9825 - val_loss: 4.5123 - val_accuracy: 0.9760 Epoch 18/50 63/63 [==============================] - 312s 5s/step - loss: 1.7373 - accuracy: 0.9795 - val_loss: 4.0130 - val_accuracy: 0.9780 Epoch 19/50 63/63 [==============================] - 314s 5s/step - loss: 1.7217 - accuracy: 0.9805 - val_loss: 4.1217 - val_accuracy: 0.9770 Epoch 20/50 63/63 [==============================] - 316s 5s/step - loss: 1.4519 - accuracy: 0.9810 - val_loss: 3.4038 - val_accuracy: 0.9770 Epoch 21/50 63/63 [==============================] - 315s 5s/step - loss: 1.7264 - accuracy: 0.9800 - val_loss: 3.6269 - val_accuracy: 0.9750 Epoch 22/50 63/63 [==============================] - 314s 5s/step - loss: 1.1591 - accuracy: 0.9815 - val_loss: 4.4193 - val_accuracy: 0.9760 Epoch 23/50 63/63 [==============================] - 313s 5s/step - loss: 0.8612 - accuracy: 0.9875 - val_loss: 3.3034 - val_accuracy: 0.9750 Epoch 24/50 63/63 [==============================] - 301s 5s/step - loss: 1.6028 - accuracy: 0.9820 - val_loss: 2.5859 - val_accuracy: 0.9810 Epoch 25/50 63/63 [==============================] - 306s 5s/step - loss: 0.9177 - accuracy: 0.9870 - val_loss: 3.0890 - val_accuracy: 0.9810 Epoch 26/50 63/63 [==============================] - 303s 5s/step - loss: 1.1154 - accuracy: 0.9830 - val_loss: 3.4198 - val_accuracy: 0.9720 Epoch 27/50 63/63 [==============================] - 305s 5s/step - loss: 1.5131 - accuracy: 0.9815 - val_loss: 3.4072 - val_accuracy: 0.9780 Epoch 28/50 63/63 [==============================] - 268s 4s/step - loss: 0.5730 - accuracy: 0.9860 - val_loss: 2.7802 - val_accuracy: 0.9780 Epoch 29/50 63/63 [==============================] - 241s 4s/step - loss: 1.1374 - accuracy: 0.9820 - val_loss: 2.1723 - val_accuracy: 0.9780 Epoch 30/50 63/63 [==============================] - 230s 4s/step - loss: 0.5363 - accuracy: 0.9850 - val_loss: 2.1879 - val_accuracy: 0.9810 Epoch 31/50 63/63 [==============================] - 231s 4s/step - loss: 0.3411 - accuracy: 0.9940 - val_loss: 2.2453 - val_accuracy: 0.9790 Epoch 32/50 63/63 [==============================] - 234s 4s/step - loss: 0.6924 - accuracy: 0.9885 - val_loss: 2.8517 - val_accuracy: 0.9750 Epoch 33/50 63/63 [==============================] - 233s 4s/step - loss: 0.9758 - accuracy: 0.9835 - val_loss: 2.9216 - val_accuracy: 0.9760 Epoch 34/50 63/63 [==============================] - 233s 4s/step - loss: 0.5718 - accuracy: 0.9865 - val_loss: 3.2723 - val_accuracy: 0.9750 Epoch 35/50 63/63 [==============================] - 237s 4s/step - loss: 1.0575 - accuracy: 0.9805 - val_loss: 2.1810 - val_accuracy: 0.9750 Epoch 36/50 63/63 [==============================] - 233s 4s/step - loss: 0.5225 - accuracy: 0.9860 - val_loss: 2.8423 - val_accuracy: 0.9710 Epoch 37/50 63/63 [==============================] - 234s 4s/step - loss: 0.4514 - accuracy: 0.9910 - val_loss: 2.3364 - val_accuracy: 0.9740 Epoch 38/50 63/63 [==============================] - 239s 4s/step - loss: 0.6142 - accuracy: 0.9905 - val_loss: 2.5889 - val_accuracy: 0.9750 Epoch 39/50 63/63 [==============================] - 235s 4s/step - loss: 0.3735 - accuracy: 0.9910 - val_loss: 2.5472 - val_accuracy: 0.9760 Epoch 40/50 63/63 [==============================] - 244s 4s/step - loss: 0.7443 - accuracy: 0.9870 - val_loss: 2.4022 - val_accuracy: 0.9770 Epoch 41/50 63/63 [==============================] - 244s 4s/step - loss: 0.6068 - accuracy: 0.9885 - val_loss: 2.4266 - val_accuracy: 0.9750 Epoch 42/50 63/63 [==============================] - 243s 4s/step - loss: 0.5422 - accuracy: 0.9880 - val_loss: 5.0947 - val_accuracy: 0.9630 Epoch 43/50 63/63 [==============================] - 254s 4s/step - loss: 0.4361 - accuracy: 0.9890 - val_loss: 3.0965 - val_accuracy: 0.9730 Epoch 44/50 63/63 [==============================] - 264s 4s/step - loss: 0.5430 - accuracy: 0.9865 - val_loss: 2.0513 - val_accuracy: 0.9750 Epoch 45/50 63/63 [==============================] - 253s 4s/step - loss: 0.5985 - accuracy: 0.9890 - val_loss: 2.7131 - val_accuracy: 0.9770 Epoch 46/50 63/63 [==============================] - 339s 5s/step - loss: 0.9335 - accuracy: 0.9840 - val_loss: 2.0440 - val_accuracy: 0.9750 Epoch 47/50 63/63 [==============================] - 1881s 30s/step - loss: 0.4963 - accuracy: 0.9865 - val_loss: 2.6356 - val_accuracy: 0.9800 Epoch 48/50 63/63 [==============================] - 241s 4s/step - loss: 0.5021 - accuracy: 0.9895 - val_loss: 2.1425 - val_accuracy: 0.9750 Epoch 49/50 63/63 [==============================] - 246s 4s/step - loss: 0.3791 - accuracy: 0.9910 - val_loss: 2.2035 - val_accuracy: 0.9770 Epoch 50/50 63/63 [==============================] - 252s 4s/step - loss: 0.7157 - accuracy: 0.9850 - val_loss: 2.0584 - val_accuracy: 0.9780
In [ ]:
accuracy = history.history["accuracy"]
val_accuracy = history.history["val_accuracy"]
loss = history.history["loss"]
val_loss = history.history["val_loss"]
epochs = range(1, len(accuracy) + 1)
plt.plot(epochs, accuracy, "bo", label="Training accuracy")
plt.plot(epochs, val_accuracy, "b", label="Validation accuracy")
plt.title("Training and validation accuracy")
plt.legend()
plt.figure()
plt.plot(epochs, loss, "bo", label="Training loss")
plt.plot(epochs, val_loss, "b", label="Validation loss")
plt.title("Training and validation loss")
plt.legend()
plt.show()
Indicating that the model is adapting to fit the training data, the training loss (loss) dramatically drops across epochs.
A steady improvement in training accuracy indicates that the model is better at accurately classifying the photos in the training set.
Explore the relative performance of the models (make sure to load the best version of each model) (2.5 points):
accuracy
confusion metric
precision, recall, F1-score,
precision-recall curve.
Explore specific examples in which the model failed to predict correctly
In [ ]:
from sklearn.metrics import confusion_matrix, precision_recall_curve, PrecisionRecallDisplay, classification_report
import matplotlib.pyplot as plt
import numpy as np
# Paths to the best models
best_model_paths = [
"C:/Users/HP/OneDrive/Desktop/LAKSHAY/models/t_model_1.keras",
"C:/Users/HP/OneDrive/Desktop/LAKSHAY/models/VGG_model_2.keras"
]
In [ ]:
from tensorflow.keras.models import load_model
# Load each model individually
for path in best_model_paths:
try:
model = load_model(path, custom_objects={'Rescaling': Rescaling})
print(f"Successfully loaded model from {path}")
except Exception as e:
print(f"Error loading model from {path}: {str(e)}")
Successfully loaded model from C:/Users/HP/OneDrive/Desktop/LAKSHAY/models/t_model_1.keras Successfully loaded model from C:/Users/HP/OneDrive/Desktop/LAKSHAY/models/VGG_model_2.keras
In [ ]:
# Create an empty list to store loaded models
loaded_models = []
# Load each model individually
for path in best_model_paths:
try:
model = load_model(path, custom_objects={'Rescaling': Rescaling})
loaded_models.append(model)
print(f"Successfully loaded model from {path}")
except Exception as e:
print(f"Error loading model from {path}: {str(e)}")
# Evaluate the models
for i, model in enumerate(loaded_models):
print(f"\nModel {i + 1} Evaluation:")
# Evaluate accuracy
test_loss, test_accuracy = model.evaluate(test_dataset)
print("Accuracy:", test_accuracy)
# Predict labels for the test dataset
y_pred = np.concatenate([model.predict(test_dataset)])
y_true = np.concatenate([y for x, y in test_dataset])
# Compute confusion matrix
cm = confusion_matrix(y_true, y_pred > 0.5)
print("Confusion Matrix:")
print(cm)
# Compute precision, recall, F1-score
report = classification_report(y_true, y_pred > 0.5, target_names=["class_0", "class_1"])
print("Classification Report:")
print(report)
# Compute precision-recall curve
precision, recall, _ = precision_recall_curve(y_true, y_pred)
plt.plot(recall, precision, marker='.')
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision-Recall Curve')
plt.show()
Successfully loaded model from C:/Users/HP/OneDrive/Desktop/LAKSHAY/models/t_model_1.keras
Successfully loaded model from C:/Users/HP/OneDrive/Desktop/LAKSHAY/models/VGG_model_2.keras
Model 1 Evaluation:
63/63 [==============================] - 10s 161ms/step - loss: 0.5657 - accuracy: 0.7270
Accuracy: 0.7269999980926514
63/63 [==============================] - 11s 168ms/step
Confusion Matrix:
[[619 381]
[599 401]]
Classification Report:
precision recall f1-score support
class_0 0.51 0.62 0.56 1000
class_1 0.51 0.40 0.45 1000
accuracy 0.51 2000
macro avg 0.51 0.51 0.50 2000
weighted avg 0.51 0.51 0.50 2000
Model 2 Evaluation:
63/63 [==============================] - 210s 3s/step - loss: 1.9934 - accuracy: 0.9760
Accuracy: 0.9760000109672546
63/63 [==============================] - 214s 3s/step
Confusion Matrix:
[[496 504]
[508 492]]
Classification Report:
precision recall f1-score support
class_0 0.49 0.50 0.50 1000
class_1 0.49 0.49 0.49 1000
accuracy 0.49 2000
macro avg 0.49 0.49 0.49 2000
weighted avg 0.49 0.49 0.49 2000
In [ ]:
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
# class names
class_names = ["cat", "dog"]
misclassified_indices = np.where(y_true != (y_pred > 0.5))[0]
# Display 5 misclassified images
num_images_to_display = min(5, len(misclassified_indices))
for i in range(num_images_to_display):
misclassified_index = misclassified_indices[i]
# predicted class label
predicted_class = class_names[int(y_pred[misclassified_index])]
actual_class = class_names[int(y_true[misclassified_index])]
print(f"Example {misclassified_index}: Predicted={predicted_class}, Actual={actual_class}")
#
img_path = f"C:/Users/HP/OneDrive/Desktop/LAKSHAY/test/{actual_class}/{actual_class}.{misclassified_index + 1500}.jpg"
try:
img = Image.open(img_path)
plt.imshow(img)
plt.title(f"Predicted: {predicted_class}, Actual: {actual_class}")
plt.axis('off')
plt.show()
except FileNotFoundError:
print(f"Error: Image file not found at path {img_path}")
Example 0: Predicted=dog, Actual=dog
Example 0: Predicted=dog, Actual=dog
Example 0: Predicted=dog, Actual=dog
Example 0: Predicted=dog, Actual=dog
Example 0: Predicted=dog, Actual=dog
Add your conclusions. (1 point)
Model : 1
With a minor improvement in the precision, recall, and F1-score for both classes, the model's overall performance is still comparable to the results of the last examination.
The accuracy stays at 72.7%.
The model performs better than before, but it may still need further research and tweaks to improve, especially when accurately classifying class 1 data.
Model : 2
Compared to Model 1, the accuracy is 0.976, which is much higher (around 0.5). This implies that the photos are classified far more accurately by Model 2.
By doing the calculation of confusion matrix The model displays similar precision, recall, and F1-score for both classes, indicating balanced performance.
With significantly better accuracy and balanced performance in all classes, Model 2 trumps Model 1, even if Model 1 still shows some progress.
As of the last part i tried using GPU and successfully did it but am unable to show that part as here because of sudeen issue with my machine so have completed this code in other machine just mentioning and will be attaching some images for that part which i did on gpu with the help of google collab.